Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage: use SetOptions() when reusing Pebble iterators #81242

Merged
merged 1 commit into from
May 23, 2022
Merged

storage: use SetOptions() when reusing Pebble iterators #81242

merged 1 commit into from
May 23, 2022
+56 −298

Conversation

erikgrinaker
Copy link
Contributor

@erikgrinaker erikgrinaker commented May 13, 2022 •
edited
Loading

Iterator reuse now relies on Pebble.SetOptions() to configure the
reused Pebble iterator. This allows a wider range of iterators to be
reused, since previously only the bounds could be changed on existing
iterators.

Release note: None

@erikgrinaker erikgrinaker self-assigned this May 13, 2022
@cockroach-teamcity
Copy link
Member

This change is Reviewable

This was referenced May 13, 2022
Closed
Closed
@erikgrinaker erikgrinaker force-pushed the mvcc-iterator-setoptions branch from 18cdc6e to 87eb356 Compare May 15, 2022 12:38
jbowens
jbowens reviewed May 16, 2022
View reviewed changes
Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up that while cockroachdb/pebble#1675 merged, the Pebble bump is blocked on resolving a breaking code change to vfs.WithDiskHealthChecks, which in turn is blocked on cockroachdb/pebble#1690. 😵

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nicktrav and @sumeerbhola)

@erikgrinaker erikgrinaker force-pushed the mvcc-iterator-setoptions branch from 87eb356 to 39cb5db Compare May 17, 2022 16:30
@erikgrinaker erikgrinaker marked this pull request as ready for review May 17, 2022 16:31
@erikgrinaker erikgrinaker requested a review from a team as a code owner May 17, 2022 16:31
@erikgrinaker erikgrinaker force-pushed the mvcc-iterator-setoptions branch 2 times, most recently from 00c73da to 3c2f325 Compare May 18, 2022 22:36
@erikgrinaker
storage: use SetOptions() when reusing Pebble iterators
0a00291 
Iterator reuse now relies on `Pebble.SetOptions()` to configure the
reused Pebble iterator. This allows a wider range of iterators to be
reused, since previously only the bounds could be changed on existing
iterators.

Release note: None
@erikgrinaker erikgrinaker force-pushed the mvcc-iterator-setoptions branch from 3c2f325 to 0a00291 Compare May 23, 2022 11:34
@erikgrinaker
Copy link
Contributor Author

This should be good to go now, PTAL.

A quick kv95 run shows a slight performance regression here. I think we can merge this anyway, and work on performance optimizations later:

master:

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  180.0s        0       19341212       107450.5      1.6      1.1      4.5     10.5    159.4  read

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  180.0s        0        1017946         5655.2      4.0      3.3      8.9     15.2    121.6  write

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__result
  180.0s        0       20359158       113105.7      1.7      1.2      5.0     11.0    159.4  

setoptions:

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  180.0s        0       18849240       104717.1      1.6      1.2      4.7     10.5    117.4  read

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
  180.0s        0         992337         5512.9      4.1      3.4      9.4     15.7    201.3  write

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__result
  180.0s        0       19841577       110230.1      1.7      1.2      5.2     11.0    201.3

@erikgrinaker erikgrinaker mentioned this pull request May 23, 2022
Merged
jbowens
jbowens approved these changes May 23, 2022
View reviewed changes
Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

I think we can merge this anyway, and work on performance optimizations later

Sounds good. I'll look at cockroachdb/pebble#1709 this week, which I think might make a difference here.

Reviewed 4 of 5 files at r4, 4 of 14 files at r5, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @erikgrinaker, @nicktrav, and @sumeerbhola)

pkg/storage/pebble_iterator.go line 117 at r5 (raw file):

		} else {
			var err error
			if p.iter, err = iterToClone.Clone(); err != nil {

Just a fyi, I filed cockroachdb/pebble#1712 so that we can avoid the SetOptions call in this case.

@erikgrinaker
Copy link
Contributor Author

Sounds good, TFTR!

bors r=jbowens

@craig
Copy link
Contributor

craig bot commented May 23, 2022

Build succeeded:

@craig craig bot merged commit 8678422 into cockroachdb:master May 23, 2022
@erikgrinaker erikgrinaker deleted the mvcc-iterator-setoptions branch May 25, 2022 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbowens jbowens jbowens approved these changes

@nicktrav nicktrav Awaiting requested review from nicktrav

@sumeerbhola sumeerbhola Awaiting requested review from sumeerbhola

Assignees

@erikgrinaker erikgrinaker

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@erikgrinaker @cockroach-teamcity @jbowens